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ABSTRACT 


This  thesis  evaluates  both  the  interface  design  process  and  the  map-based  mission 
planning  tools  of  the  Loosely  Coupled  Components  Research  Group,  Naval  Postgraduate 
School  for  human  factors  usability.  After  identifying  flaws  in  the  process  and  usability 
problems  in  the  interface  designs,  a  new  software  design  process  and  map-based  mission¬ 
planning  tool  are  developed.  A  usability  study  was  conducted  on  the  new  mission¬ 
planning  tool,  determining  it  to  be  a  usable  product  while  establishing  baseline  data  for 
future  interface  improvements.  The  map-based  mission-planning  tool,  written  in  the  Java 
programming  language,  is  called  the  Mapping,  Information,  Display,  and  Analysis 
System  (MIDAS).  In  its  Beta  form,  MIDAS  can  display  any  geo-referenced  map  or 
image  and  allow  users  to  annotate  it  with  several  graphical  tools.  Future  versions  will 
incorporate  existing  map-based  decision-aiding  tools  such  as  optimal  track  routing, 
intelligence  image  rubber-sheeting,  and  wirelessly  networked  unit  tracking.  This  thesis 
recommends  the  incorporation  of  human  factors  early  in  the  software  design  process  and 
quality  usability  studies  on  interfaces  to  ensure  a  usable  product. 
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errors,  they  cannot  be  considered  validated.  Any  application  of  these  programs  without 
additional  verification  is  at  the  risk  of  the  user. 
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EXECUTIVE  SUMMARY 


Joint  Vision  2010  establishes  a  conceptual  template  for  leveraging  technological 
opportunities  to  achieve  new  levels  of  effectiveness  in  joint  warfighting  capabilities  for 
the  United  States  military.  Of  JV2010’s  five  areas  of  focus,  Information  Superiority  is 
the  most  affected  by  emerging  computer,  communication,  micro-miniaturization,  and 
Internet  technologies.  Information  Superiority  will  play  a  dominant  role  in  the  future  of 
warfare.  Technological  advances  in  satellite  imagery,  remotely  piloted  vehicles,  mobile 
communications,  the  Internet,  and  GPS  alone  have  already  swamped  military  leaders 
with  more  information  than  they  can  effectively  use  in  a  tactical  situation.  The  ability  to 
collect,  analyze,  and  disseminate  vast  quantities  of  useful  information  is  Joint  Vision 
2010’s  primary  vehicle  to  achieve  “Battlespace  Dominance.” 

The  Department  of  Defense  has  responded  to  Joint  Vision  2010’ s  technological 
challenge  with  various  advanced  warfighting  programs  like  the  Command  Post  of  the 
Future  project,  the  Land  Warrior  2000  project,  and  the  Naval  Postgraduate  School 
Loosely  Coupled  Components  Research  Group.  All  three  programs  employ  Internet-time 
technologies  in  wireless  networking,  command  and  control,  and  mission  planning. 

How  do  military  leaders  know  they  are  getting  the  biggest  technological  bang  for 
their  research  dollar?  How  do  they  know  if  the  newest  techno-information  system  is 
overwhelming  their  commanders  or  giving  them  the  Information  Superiority  they  need  to 
win  the  battle?  The  answer  to  these  questions  is  based  in  Human  Factors  and  the  Human 
Computer  Interface.  Whether  it  is  a  laptop  computer  screen,  a  remote  imaging  device,  or 
a  computer  aided  rifle-sight,  humans  are  interacting  with  a  computer.  No  matter  how 
powerful  the  interface  appears,  the  sailor  or  soldier  operating  it  must  be  considered  in  its 
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design.  Technological  advances  are  only  moving  in  one  direction  -  smaller,  faster,  and 
more  complicated.  Knowing  that  military  research  and  development  has  fully 
incorporated  the  user’s  needs  into  interface  design  ensures  a  useful,  powerful,  and 
effective  information  system. 

This  thesis  evaluates  the  human  computer  interface  of  existing  map-based 
mission  planning  tools  developed  by  a  faculty  and  student  research  group  in  the  Naval 
Postgraduate  School’s  Operations  Research  Department.  After  careful  human  factors 
based  analysis,  various  usability  flaws  were  identified  and  an  improved  Graphic  User 
Interface  was  designed.  The  new  interface,  written  in  Java,  utilizes  any  imagery  that  can 
be  geo-referenced  and  will  soon  incorporate  numerous  operations  analysis  decision  tools 
for  the  military  planner.  A  usability  study  was  conducted  and  results  were  compared 
with  industry  and  the  DoD  standards  for  usable  interfaces.  The  new  map-based  planning 
tool’s  interface  was  determined  to  be  usable  and  therefore  was  accepted  as  the  base 
interface  for  further  operations  analysis  decision-making  tools.  Not  only  is  the  new 
graphic  user  interface  a  DoD  model  for  Java  map-based  mission  planning  software,  but 
the  Loosely  Coupled  Components  research  group  is  leading  the  way  in  military  software 
development  that  incorporates  human  factors  in  the  software  design  process. 
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I. 


INTRODUCTION 


The  module  is  a  scale  of  proportions  that  makes  the  bad  difficult  and  the 
good  easy. 

-  Albert  Einstein  to  Le  Corbusier  (1964) 
referring  to  intellectual  model  representation 


A.  PROBLEM 

Joint  Vision  2010  establishes  a  conceptual  template  for  leveraging  technological 
opportunities  to  achieve  new  levels  of  effectiveness  in  joint  warfighting  for  the  United 
States  military  (CJCS,  1996).  Of  JV2010’s  five  areas  of  focus,  Information  Superiority  is 
most  affected  by  emerging  computer,  communication,  micro-miniaturization,  and  internet 
technologies. 

The  Information  Superiority  concept  exploits  advances  in  collection,  processing, 
and  dissemination  technologies  to  “mitigate  the  impact  of  the  friction  and  fog  of  war,” 
while  at  the  same  time,  denying  the  enemy  the  right  to  the  same  (CJCS,  1996). 
Information  Superiority  is  achieved  by  fusing  and  processing  information  from  all 
available  intelligence  sources  and  disseminating  a  usable  product  to  thousands  of 
locations  in  a  timely  manner.  Using  superior  information,  our  joint  fighting  forces  will 
achieve  “Dominant  Battlespace  Awareness”  allowing  increased  force  dispersion, 
mobility,  and  lethality  (CJCS,  1996). 

The  Department  of  Defense  has  responded  to  Joint  Vision  2010’s  Information 
Superiority  call  with  several  technology  based  research  programs.  Some  examples  are 
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DARPA’s  Command  Post  of  the  Future  (CPOF),  the  Army’s  Land  Warrior  2000  program 
(LW2K),  and  the  Naval  Postgraduate  School’s  Loosely  Coupled  Components  (LCC) 
research  group. 

The  LCC  research  combines  several  Operations  Research  based  tools  to  aid  in 
map  based  mission  planning.  Some  of  the  tools  are:  National  Imagery  and  Mapping 
Agency  (NIMA)  formatted  map  display,  intelligence  imagery  rubber-sheeting,  route¬ 
planning,  shortest-path  decision  aids,  whiteboard-style  map  annotations,  battlefield 
training  monitors,  and  field-deployed  database  servlets.  Though  each  is  an  effective 
stand-alone  tool,  when  fused  into  one  interface  they  become  ineffective  and  unusable 
beyond  the  academic  environment.  The  fault  in  the  software’s  poor  usability  lies  in  its 
outdated  software  design  process.  In  order  to  make  the  software  more  usable,  the 
software  design  process  must  be  improved  and  adapted  to  industry-wide  Human 
Computer  Interface  (HCI)  design  standards.  After  examining  the  existing  software  under 
the  new  design  process,  it  was  determined  by  the  LCC  research  group  that  the  only 

*  I 

acceptable  approach  was  to  design  a  new  map-based  mission-planning  tool  from  the 
ground  up.  This  thesis  proposes  a  replacement  to  the  existing  LCC  map-based  mission¬ 
planning  tool  and  developed  a  human  factors  software  design  methodology  for  future 
LCC  Operations  Research  systems. 

B.  HUMAN  FACTORS  AND  GUI  DESIGN 

Information  Superiority  based  in  technological  advances  is  achieved  when  a 
human  is  able  to  successfully  use  a  computer  to  collect,  process,  and  disseminate 
information.  Each  DoD  research  group  is  developing  its  own  graphic  user  interface 
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(GUI)  to  manage  these  processes.  Their  GUI’s  will  allow  soldiers,  sailors,  airmen, 
marines,  medics,  commanders,  or  SEAL’S  to  interact  with  their  computers  to  exchange 
information  with  any  computer.  Though  technology  and  software  electronically  bridge 
gaps  between  dissimilar  computer  systems,  process  large  quantities  of  information,  and 
adapt  to  unknown  hardware,  the  soldier  is  the  part  of  the  equation  that  puts  the 
technology  to  use.  If  she  or  he  is  bogged  down  wrestling  with  menu  structure  or  is 
unfamiliar  with  applications  of  his  software,  no  technology  can  lead  to  success.  A  strong 
set  of  Human  Computer  Interface  (HCI)  guidelines  is  needed  to  ensure  the  soldier’s 
information  display  system  has  been  designed  with  his  tasks  and  needs  in  mind. 

1.  Graphic  User  Interfaces 

The  success  of  Microsoft  Windows®  and  the  Macintosh®  O/S  as  popular  Graphic 
User  Interfaces  is  based  on  their  designers’  strong  incorporation  of  the  Mental  Models 
and  Metaphors  methods  as  discussed  by  Wickens,  Gordon,  and  Liu  (1998).  The  Mental 
Models  method  is  a  dynamic  model  of  the  user’s  knowledge  of  the  following:  system 
components,  how  the  system  works,  how  components  are  related,  what  the  internal 
processes  are,  and  how  the  user  affects  the  components.  The  Metaphor  method  is  the 
process  of  using  objects  and  events  in  a  software  system  that  are  taken  from  a  non¬ 
computer  domain  such  as  “desktops,”  “cut  and  paste,”  and  “trash  cans”  (Wozny,  1989). 

Mayhew  (1992)  states  that  designers  should  enable  the  user  to  develop  an 
effective  Mental  Model.  An  effective  mental  model  is  one  in  which  the  user  can  mentally 
represent  the  relationships  between  or  perform  actions  on  working  components  of  the 
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GUI.  Wickens,  Gordon  &  Liu  (1998)  provide  four  suggestions  to  improve  the  mental 
model,  (1)  Make  invisible  parts  and  computing  visible  to  the  user  (i.e.  dragging  a  file  to  a 
trash  can  to  delete  it),  (2)  Provide  feedback  to  the  user  (i.e.,  showing  statuses  of  loading, 
saving,  printing),  (3)  Build  in  consistency  (i.e.,  established  patterns  and  rules  common 
across  applications),  and  (4)  Present  functionality  through  a  familiar  metaphor  utilizing 
real  world  analogies  (i.e.,  physically  moving  a  mouse  pointer  through  the  non-physical 
environment  of  a  computer  display). 

The  Metaphor  Method,  the  second  half  of  effective  GUI  design,  provides  the  user 
with  familiar  metaphors  for  completing  tasks.  One  example  is  the  ability  of  World  Wide 
Web  users  to  chat  on  the  Internet  in  a  “room.”  Though  none  of  the  actions  physically 
occur,  users  can  identify  with  the  metaphor  of  entering  a  room  full  of  people  and  chatting 
with  one  or  all  of  them.  Other  less  obvious  metaphors  include  matrix-structured 
spreadsheets,  desktops,  clocks,  calendars,  and  back/forward  icons  for  turning  “pages”  of 
virtual  books,  manuals,  or  Web  pages. 

Interface  designers  must  also  be  careful  using  metaphors  that  are  also  vulnerable 
to  enrors.  Differences  between  the  metaphorical  world  and  the  software  system,  if  not 
made  explicit,  can  cause  errors  or  gaps  in  the  user’s  mental  models  of  the  software 
system  (Halasz  &  Moran,  1982).  Examples  of  metaphor  error  are  turning  pages  left  and 
right  in  a  virtual  book  by  using  Page  Up  /  Page  Down  keys  or  pressing  the  MS®  “Start” 
button  to  initiate  a  computer  shutdown. 

GUI  technology  and  design  are  not  limited  to  conventional  keyboards,  monitors, 
and  speakers.  GUI  design,  based  strongly  on  Mental  Models  and  Metaphors,  is  crucial  to 
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implementing  successful  present  and  future  software  applications.  ‘Thirty-seven  to  fifty 
percent  of  [industry]  efforts  throughout  the  software  life  cycle  are  related  to  the  system’s 
user  interface”  (Hefley,  Buie,  Lynch,  Muller,  Hoecker,  Carter,  and  Roth,  1994,  p.315). 
Financial  implications  of  these  efforts  force  software  development  companies  to  join 
human  factors  engineers  and  software  programmers  at  the  onset  of  system  design.  Their 
goal  is  to  reduce  the  short  and  long-term  costs  associated  with  poor  design.  As 
computers  and  displays  become  smaller,  GUI’s  will  become  ever  more  important  and 
will  be  relied  upon  to  maintain  the  information  bandwidth  required  to  complete 
complicated  tasks.  Task  completion,  however,  is  not  the  only  yardstick  for  declaring  a 
GUI  design  successful.  The  experiments  in  usability  measure  the  quality  and 
effectiveness  of  GUI  designs. 

2.  Usability 

The  key  to  a  successful  GUI  is  whether  or  not  it  is  designed  with  human  usability 
as  its  primary  goal.  In  the  past  decade,  computer  use  has  expanded  to  toddlers  and  the 
elderly  and  from  making  scientific  calculations  to  writing  e-mail  and  joining  virtual 
combat  missions.  Software  companies  can  no  longer  afford  to  push  highly  technical  and 
unfriendly  software  on  customers.  To  stay  in  business,  they  are  meeting  these  widening 
demands  through  extensive  usability  research  and  adherence  to  HCI  guidelines. 
According  to  Nielsen  (1993),  software  usability  is  traditionally  associated  with 
leamability  (ability  to  quickly  become  productive  with  software),  efficiency  (high 
productivity  after  initial  learning  period),  memorability  (relearning  is  not  necessary  after 
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periods  of  non-use),  errors  (low  error  rate,  ease  of  error  recovery,  no  catastrophic  errors), 
and  satisfaction  (subjectively  determined). 

There  are  many  ways  to  measure  whether  a  Graphic  User  Interface  meets  the 
minimum  requirements  in  an  established  set  of  standards.  Each  interface  requires  a 
different  testing  method.  The  most  commonly  used  methods  to  measure  usability  are 
number  of  errors,  time  to  perform  tasks,  and  user  subjective  reactions. 

3.  Usability  Testing 

The  overall  goal  of  interface  usability  testing  is  to  identify  and  rectify  usability 
deficiencies  in  computer-based  human  computer  interaction  (Rubin,  1994).  There  are 
many  measures  of  a  GUI’s  usability.  Listed  here  are  a  few  used  by  Microsoft’s®  usability 
labs:  benchmark  studies,  heuristic  reviews,  task  analyses,  error  analyses,  and  competitive 
studies  (Microsoft,  1998).  Nielsen  (1999)  explains  the  simplest  usability  metric  is 
success  rate,  which  is  the  best  estimate  of  the  true  success  rate  for  a  similar  population 
user.  In  order  to  determine  if  a  success  rate  is  acceptable,  standards  must  be  established 
prior  to  usability  testing.  Either  benchmarks  from  previous  studies  or  heuristic  industry 
standards  provide  the  ruler  for  newly  collected  data  (Nielsen  and  Molich,  1990,  Nielsen 
1994).  Industry  standards  for  icon  identification  success  rates  average  70%  for  initial 
exposure  and  100%  there  after  (Bickford,  personal  communication,  11  May  2000). 
Standards  for  acceptable  task  analysis  success  rates  are  90%  (Bickford,  personal 
communication,  1 1  May  2000). 
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4.  Human  Computer  Interaction  Guidelines 

In  order  to  make  HCI  successful  across  all  platforms  and  software,  interface 
guidelines  must  be  thoroughly  integrated  into  the  product  realization  process  (Lund  & 
Tschirgi,  1991).  But  before  HCI  and  the  production  processes  can  be  integrated,  HCI 
must  be  well  understood. 

Dix,  Finlay,  Abowd  &  Beale  (1998)  define  HCI  by  breaking  it  into  three  parts. 
The  human  user  is  any  individual  or  group  of  users  in  an  organization  participating  in 
task  or  process  completion.  The  computer  is  any  technology  such  as  a  palmtop,  laptop, 
desktop,  mainframe,  or  process  control  system.  Interaction  is  any  direct  or  indirect 
communication  between  a  user  and  a  computer  to  accomplish  tasks.  Since  the  early 
advent  of  Macintosh’s®  desktop,  Microsoft’s®  Windows  series,  and  the  vast  array  of 
Internet  browsers,  human  factors  experts  have  been  applying  their  expertise  to  HCI. 

It  is  also  important  to  understand  that  common  philosophical  HCI  guidelines 
serve  as  a  guide  and  base  to  all  interface  designers,  no  matter  what  company  mandated 
standards  are  in  place  (Hix  &  Hartson,  1993).  HCI  guidelines  are  not  limited  to  blue  chip 
companies  and  Silicon  Valley  software  developers,  either;  the  European  community  has 
also  recognized  the  requirement  for  common  guidelines. 

In  response  to  the  need  for  common  visual  display  terminals  (VDT’s)  in  the 
European  Banking  and  Economic  Area,  European  Community  members  transformed  a 
previously  human  factors  related  “minimum  safety  and  health  requirements  directive” 
into  national  law.  They  have  gone  one  step  further  by  requiring  software  developers. 
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who  may  lack  knowledge  in  the  area  of  human  factors,  to  utilize  design-aid  software 
tools  that  incorporate  built-in  human  factor  guides  and  testing  criteria  (Reiterer,  1993). 

Even  after  subscribing  to  HCI  guidelines,  some  experts  believe  that  we  have  not 
evolved  from  our  early  non-GUI  days  of  computing.  Raskin  (1997)  explains  that  our 
present  systems  are  as  large,  complex,  and  nightmarish  as  the  mainframes  they  first 
displaced;  he  adds  that  to  be  a  “power-user”  one  is  expected  to  know,  on  average,  over 
three  hundred  settings  of  the  system  he  is  using. 

C.  BACKGROUND 

1.  LCC  Software  Design  Processes 

The  previous  Loosely  Coupled  Components  Software  design  process  was 
simple.  When  a  new  idea  for  a  map-based  OR  tool  was  discovered,  it  was  immediately 
coded  for  proof-of-concept.  The  process  allowed  little  or  no  human  factors  application 
until  after  the  code  was  shown  to  work.  The  outcome  of  this  process  has  been  a  sound 
OR  tool  that  only  a  few  people  can  utilize.  Some  of  the  user  difficulties  are  discussed 
later  in  this  chapter.  Figure  1  shows  the  design  process  used  prior  to  this  thesis. 
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Figure  1.  Existing  LCC  Software  Design  Process 

The  key  to  a  successful  and  usable  piece  of  software  is  incorporating 
human  factors  at  the  start  of  the  design  process.  Modeled  after  Lim  and  Long’s  (1992) 
Structured  Human  Factors  Design  Framework,  figure  2  shows  an  adapted  version  for  the 
LCC  research  group. 
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2.  Current  Information  Display  Systems  Research 

The  Defense  Advanced  Research  Projects  Agency  (DARPA)  is  researching  ways 
to  provide  an  integrated  display  and  analysis  tool  to  military  commanders  to  aid  decision 
making  abilities  via  the  Command  Post  of  the  Future  (CPOF)  program.  The  Land 
Warrior  2000  project  is  investigating  wirelessly  networked  wearable-computers  and 
associated  displays  for  use  by  individual  soldiers  to  increase  their  combat  effectiveness. 
The  LCC  project  is  developing  methods  to  loosely  connect  various  Commercial  Off-The- 
Shelf  (COTS)  components  to  create  military  systems  for  mission  planning  and  execution 
that  contain  the  capabilities  envisioned  by  Joint  Vision  2010. 

a)  DARPA ’s  Command  Post  of  the  Future 

DARPA’ s  CPOF  project  is  to  develop  advanced  technology  to  create  an 
adaptive,  decision-centered,  visualization  environment  for  the  future  commander  with  the 
end  goal  of  doubling  the  speed  and  quality  of  command  decisions  while  cutting  the 
required  support  staff  in  half  (Page,  2000).  Page,  project  manager  of  the  CPOF  program, 
further  states,  “As  current  technology  floods  the  military  commander  with  messages, 
images,  and  data,  he  will  require  larger  staffs  and  more  computers  to  process,  interpret, 
integrate,  and  understand  the  incoming  information  streams  (Page,  2000,  p.l).” 

Recognizing  the  human  difficulty  in  processing  large  volumes  of 
information  at  high  rates,  DARPA  is  incorporating  various  advances  in  Human  Computer 
Interaction  (HCI)  technologies  into  its  design. 

Some  of  these  technologies  include  GUI-based  3D  visualization, 
interactive  3D  techniques,  Natural  Language  processing,  and  Knowledge  Base  querying 
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(Despain  &  Westervelt,  1997).  They  are  also  investigating  human-computer  interfaces 
that  go  beyond  current  GUI  technology.  Some  examples  include  the  creation  of  a  “cyber¬ 
warrior,”  or  computer-enhanced  soldier,  who  could  utilize  visual  cortex  implants, 
vibration,  temperature,  eye-trackers,  voice  control,  data  gloves  and  intelligent  user 
interfaces. 


b)  U.S.  Army ’s  Land  Warrior  2000 

In  1991,  an  Army  Science  Board  Study  recommended  that  the  soldier  be 
treated  as  a  “complete  fighting  system;”  this  recommendation  resulted  in  the  initiative 
known  as  the  Soldier  Integrated  Protective  Ensemble  (SIPE).  After  proof  of  concept,  the 
SIPE  Program  evolved  into  the  Land  Warrior  (LW)  program  in  July  1995  and  has  since 
become  the  Land  Warrior  2000  (LW2K)  program.  The  LW2K  program  integrates  a 
computer  and  a  soldier  into  one  networked  fighting  system.  By  combining  advances  in 
computer  and  communications  technology,  inexpensive  COTS  hardware,  and  advanced 
weapons  aiming  systems,  the  Army  plans  to  employ  each  networked  soldier  as  a 
complete  weapons  platform.  (Jette,  1999) 

The  soldier  will  access  his  computer-enhanced  system  via  the  GUI  on  his 
handheld  flat  panel  display  or  its  near-future  replacement,  a  helmet-mounted  monocle. 
The  Army  plans  to  display  digital  messages,  video,  thermal  site  imagery,  graphics, 
warning  messages,  and  navigation  information  on  either  the  monocle  or  the  handheld 
panel. 
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c)  NPS’s  Loosely  Coupled  Components 

The  Naval  Postgraduate  School’s  Operations  Research  Department  began 
researching  CPOF  ideas  and  Land  Warrior’s  networking  concepts  in  1996  with  an 
Operations  Research  look  at  decision  aids,  electronic  cartography,  and  security.  The 
primary  goal  of  the  LCC  project  is  to  design,  develop,  and  demonstrate  decision  support 
systems  for  military  planning,  execution,  and  training  using  COTS  technology  from  the 
fields  of  wireless  networking,  Java®-based  object-oriented  programming,  portable 
information  display  systems,  war-fighting  training  systems,  and  mission  planning  tools 
(Bradley,  Buss,  &  Shaw,  1998).  These  LCC  objectives  are  achieved  through  a  powerful 
object-oriented  software  concept  to  which  new  software  modules  and  COTS  hardware 
can  be  added  and  removed  seamlessly  via  the  Internet  or  wireless  LAN.  Some  of  the 
COTS  equipment  includes  Palm  IIIx™  PDA’s,  Casseopias™,  Libretto™  palmtops,  bar¬ 
code  readers.  Lucent™  WAVELAN  cards,  and  global  positioning  systems. 

The  power  of  object-oriented  programming  using  the  Java  programming 
language  lies  in  platform  independence  and  dynamic  loading.  Platform  independence 
allows  software  to  be  “written  once,  [and]  run  anywhere”  (Linden,  1997).  Dynamic 
loading  allows  even  the  least  capable  computer  to  conserve  memory  by  downloading  and 
running  software  only  when  needed  and  then  purging  it  upon  completion. 

Platform  independence  and  dynamic  loading  are  relatively  new  in  military 
software  design.  Current  military  software  design  does  not  incorporate  platform 
independence.  It  relies  on  a  team  of  contracted  software  engineers  who  develop  different 
versions  of  the  same  software  tailored  to  meet  the  varying  computing  requirements  of  the 
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military  user.  The  Aegis  computer  software  on  U.S.  Navy  cruisers  and  destroyers  best 
exemplifies  this  method  of  software  design.  As  the  ship’s  combat  computer  hardware  is 
upgraded  in  staggered  fashion  throughout  the  fleet,  it  requires  a  new  software  baseline 
version.  These  differences  in  software  baselines  can  limit  the  interoperability  of  ships 
employing  the  Cooperative  Engagement  Capability  (CEC),  which  shares  fire-control 
information  between  ships  for  ballistic  missile  engagement. 

Dynamic  loading  is  also  an  under-utilized  concept  in  the  military.  The 
LW2K  program  will  be  the  first  to  incorporate  the  ability  for  a  minimally  capable  system 
to  download  programs,  similar  to  Internet  applets,  as  needed  from  network  databases  and 
then  to  purge  them  upon  task  completion.  Examples  of  possible  programs  include 
optimal  track  routing,  logistics  calculators,  and  automated  re-supply  software. 

Combining  the  strengths  of  platform  independence  and  dynamic  loading 
via  object-oriented  programming  is  the  first  step  to  creating  a  powerful  network  of 
tactical  military  computers.  The  second  step  is  a  local  area  network.  Both  the  LW2K 
and  LCC  project  are  investigating  and  utilizing  current  wireless  network  capabilities. 
The  tactical  benefits  of  wireless  networks  are  high  mobility,  encrypted  information  flow, 
and  seamless  LAN-entry  and  -exit  of  portable  systems.  The  following  are  just  a  few  of 
the  possible  information  flows  the  LCC  project  incorporates:  updated  unit  positions, 
current  orders,  and  decision  aids  to  acquire  locations  of  nearest  supply  and  medical 
stations.  None  of  these  capabilities  can  be  utilized  without  a  reliable  wireless  network 
connecting  up  to  hundreds  of  portable  computers  and  a  robust  visual  interface  to  put  the 
information  at  the  soldier’s  fingertips. 
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D.  EXISTING  LCC  SOFTWARE 


Loosely  Coupled  Components  (LCC)  software  was  developed  through  a  series  of 
Operations  Research  (OR)  Master’s  Theses  (Bilyeu,  1998;  Hattes,  1999;  Schrepf,  1999). 
As  each  student  developed  a  new  OR  tool,  their  module  was  attached  to  the  existing  LCC 
software  through  a  common  button  bar.  The  overarching  software  hub  is  Thistle. 

1.  Thistle 

Schrepf  (1999)  developed  Thistle  to  simulate  and  model  movement  of  ground 
forces  which  assists  commanders  make  decisions  for  routing  of  convoys.  The  three 
primary  GUI  design  flaws  identified  in  Thistle  are:  poor  menu  structure  and 
lconongraphy,  poor  use  of  Screen  Real  Estate,  lack  of  Positional  Constancy. 

Schrepf  s  interface  was  designed  as  a  programmer’s  device  to  incorporate  various 
Java  classes  into  one  program.  While  it  allowed  users  to  initiate  numerous  Operations 
Research  modules  by  pressing  their  associated  buttons,  it  was  not  designed  for  the 
untrained. 


EgTHISTLE  5.0  Beta  [cpm] 
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Figure  3.  Thistle’s  button  bar  continued  to  grow  as  new  tools  were  added  to  the  program. 

Every  button  has  a  text  descriptor  in  place  of  a  graphic  icon.  The  human  factor 
(HF)  flaw  in  this  design  is  two-fold.  First,  the  user  must  read  through  the  text  of  many  of 
the  buttons  prior  to  finding  the  desired  module.  A  better  design  would  be  to  place  the 
text  in  drop-down  menus  or  use  metaphorical  icons.  Second,  the  button  bar  is  not 
organized  by  task  or  by  commonality.  Some  of  the  modules  use  maps,  even  though  there 
is  a  “Map”  button  on  the  bar.  Once  again,  a  multi-leveled  drop-down  menu  would  have 
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been  a  better  choice  by  organizing  related  modules  in  the  same  menu  structure  (Minasi, 
1994).  This  menu  should  provide  the  basic  structure  of  the  software  to  the  user  without 
having  to  move  a  mouse  or  press  a  key.  Thistle’s  menu  structure  is  not  intuitive  to  the 
user.  Without  any  previous  knowledge  of  what  some  of  these  buttons  do,  a  novice  user  is 
left  to  experiment  with  each  button  until  the  desired  feature  appears. 


If  the  user  initiates  several  applications  from  Thistle’s  button  bar,  the  computer 
monitor  display  becomes  crowded  and  confusing  as  shown  in  figure  4. 


Figure  4.  Thistle’s  Display  with  all  features  operating. 


Figure  4  shows  the  second  HF  design  flaw  in  the  new  software.  Screen  real 
estate,  as  many  HF  experts  call  it,  is  poorly  managed  in  Thistle.  Not  only  are  there 
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overlapping  windows,  but  between  the  non-overlapped  ones  lie  large  areas  of  wasted 
space.  The  largest  cause  for  the  misused  real  estate  is  the  software’s  ever-expanding 
interface  design.  Every  new  module  opens  its  own  window  and  piece  of  the  user’s  visual 
field.  A  single  or  doubled  window  design  would  have  be  a  cleaner  and  less  confusing 
way  of  displaying  data  to  the  user. 

The  third  interface  design  flaw  was  the  lack  of  Positional  Constancy.  Positional 
constancy  means  the  user  can  expect  to  find  the  same  interface  layout  every  time  it  is 
executed.  Wickens,  Gordon  and  Liu  (1998)  believe  it  is  either  through  repetition  and/or 
English  language  reading  styles  that  users  expect  to  begin  any  software  application  in  the 
upper  left-hand  comer  of  the  screen.  Thistle  utilizes  multiple  windows  in  various  orders 
and  positions  that  force  the  user  to  mentally  track  numerous  module  locations  and 
eventually  minimize  and  maximize  module  windows  to  locate  them.  The  only  result  for 
novice  users  is  frustration,  confusion,  and  errors. 

2.  Flora 

The  second  primary  module  in  Thistle  is  a  dynamic  map  and  overlay  display  tool 
named  Flora  that  allows  users  to  plan  tactical  missions,  analyze  networking  problems, 
plan  convoy  routing  and  even  monitor  GPS-networked  units.  Several  tools  are  available 
to  manipulate  the  displayed  maps  and  overlays.  One  tool  was  the  zoom-in  /  zoom-out 
feature  available  in  the  pull  down  menu  that  replaces  the  active  National  Imagery  and 
Mapping  Agency  (NIMA)  map  with  the  next  larger  or  smaller  scaled  image  of  the  same 
geographic  location.  A  “Grease  Pen”  on-map  annotation  tool  is  available  for  planning 
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and  analysis  mark-ups.  Users  also  have  the  option  to  load  prepared  overlays  or  create 
new  ones  with  tools  available  in  Flora.  Overlays  can  consist  of  Grease  Pen  annotations, 
networking  graphs  with  available  networking  algorithms,  or  unit  symbology  annotations 
for  organic  and  multinational  forces. 

Flora’s  interface  was  the  Loosely  Couple  Components’  (LCC)  first  real  start  at 
incorporating  map-based  mission-planning  features  into  one  GUI.  Many  of  its  GUI 
design  flaws  limit  its  ability  to  be  widely  used  by  non-expert  users.  Some  of  it  design 
flaws  are  cumbersome  zoom-in  /  zoom-out  features,  unclear  button  functions,  poor  map 
field  of  view  and  scope,  and  poor  map  re-centering  or  dragging. 

An  example  of  Thistle’s  mapping  display  Flora  is  shown  in  figure  5. 
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Figure  5.  FLORA’S  map  display  was  the  first  tool  to  include  map  annotation  tools.  The 
“Grease  Pen”  button  launches  a  window  with  some  basic  annotation  tools. 


3. 


SOFLCC 


The  third  major  module  in  Thistle  is  SOFLCC  which  was  developed  to  implement 
a  platform  independent  mission  planning  and  analysis  system  for  the  United  States 
Special  Operations  Command  (USSOCOM)  (Bilyeu,  1998).  SOFLCC  combined  the  map 
display  capabilities  of  Flora  but  added  another  unique  feature  -  fading.  The  feature 
allowed  the  user  to  fade  a  map  into  an  underlying  satellite  image  that  had  been  either  geo- 
referenced  at  the  same  scale  or  “rubber-sheeted.”  The  term  rubber-sheet  involves 
acquiring  recognizable  landmarks  on  the  map,  the  same  landmarks  in  an  image,  and  then 
through  mathematical  algorithms,  stretch  one  or  the  other  until  they  are  synchronous. 
The  map  in  figure  6  has  been  partially  “Dissolved”  into  an  underlying  satellite  image. 
Dark  patches  are  forested  areas  corresponding  to  the  contoured  hills  on  the  map. 


Figure  6.  Special  Operations  Forces  Loosely  Coupled  Components  added  the  feature  of 
blending  satellite  imagery  with  maps  as  shown  here. 
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As  with  Flora,  SOFLCC’s  design  flaws  lie  in  weak  iconography  and  multiple 
windowing,  and  poor  map  manipulation.  The  user  has  to  read  the  unevenly  sized  buttons 
until  he  finds  the  right  one  on  the  bar  every  time.  A  simple  metaphorical  graphic  could 
increase  the  speed  and  accuracy  of  their  use  and  improve  user  leamability. 

According  to  Nolan  (1989),  icons  need  to  be  concrete-familiar  (non-confusing 
and  common)  as  opposed  to  abstract  and  unfamiliar  (confusing  and  uncommon).  Text- 
filled  tool-buttons  in  SOFLCC  are  concrete-unfamiliar.  Novice  users  can  read  the 
function  of  the  buttons,  but  may  not  inherently  know  how  to  use  them  to  accomplish  a 
task. 

Windowing  again  is  a  problem  with  SOFLCC.  When  the  Show  Overlays  button 
is  pressed,  another  window  opens  displaying  all  current  overlays  opened.  And  from  that 
window,  the  user  may  choose  from  text-labeled  buttons  to  “Hide  Layers,”  “Show 
Layers,”  “Remove  Layers,”  or  “Run  Algorithm.”  If  “Run  Algorithm”  is  pressed,  another 
window  opens  to  let  the  user  load  an  algorithm.  It  is  a  complete  surprise  to  a  novice  user 
that  algorithms  can  even  be  run  in  SOFLCC  or  that  they  can  be  reached  via  the  overlay 
buttons  since  there  is  no  indication  of  such  in  the  opening  screen’s  layout. 

SOFLCC  fares  even  worse  in  its  map  manipulation.  In  short,  there  is  none.  The 
map  itself  is  an  overlay  with  no  zoom  capability  at  all.  The  user  has  no  way  of  using  the 
map  for  any  purpose  other  than  prepared  overlays  and  algorithms. 

4.  LCC  Software  Usability  Summary 

Each  piece  of  software  was  examined  under  the  established  guidelines  set  by  the 
DoD  HCI  Style  Guide  (1994)  and  a  variety  of  compilations  by  Schneiderman  (1998), 
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Mayhew  (1992),  and  Brown  (1989).  Due  to  time  limitations,  only  Iconography  and  Map 
Manipulation  Flaws  will  be  addressed  in  the  interface  redesign.  All  of  the  Operations 
Analysis  tools  developed  in  both  Flora  and  SOFLCC  have  proven  necessity  in  military 
planning  and  operations  and  will  be  coded  into  the  new  interface  as  time  permits. 

E.  PURPOSE  AND  RATIONALE 

This  thesis  proposes  a  software  design  process  and  a  GUI  designed  under  widely 
accepted  HCI  guidelines,  to  replace  the  existing  LCC  software.  It  is  not  the  author’s 
attempt  to  weaken  the  underlying  power  of  any  of  the  LCC  programs,  but  strengthen 
them  by  folding  them  into  an  interface  that  is  easy  to  learn,  easy  to  use,  and  incoxporates 
the  features  of  all  the  programs.  A  usability  study  was  conducted  on  the  new  GUI  to 
determine  its  usability  and  to  establish  baseline  data  for  comparison  with  future  interface 
improvements. 

The  new  system,  Mapping,  Information,  Display  and  Analysis  Systems  (MIDAS) 
exploits  the  same  Java  features  the  existing  LCC  software  does,  while  maintaining 
networked  map-based  mission  planning  tools.  Due  to  the  vast  changes  in  the  interface 
design,  a  comparative  study  between  MIDAS  and  the  old  interfaces  was  not  feasible  and 
deemed  unnecessary.  Therefore,  the  study’s  scope  was  narrowed  to  the  two  major  GUI 
improvements  and  their  corresponding  usability  metrics:  Iconography  Recognition  and 
Map  Manipulation.  The  usability  study  measured  leamability,  memorability,  and 
efficiency.  It  was  hypothesized  that  MIDAS  graphical  user  interface  would  be  superior 
to  the  existing  LCC  graphical  user  interface  software  due  to  the  adherence  to  human 
factors  principles.  Specifically,  MIDAS  incorporated  drop-down  menus  and 
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metaphorical  icons  to  aid  user  readability,  standardized  mouse  functions  across  all  map- 
manipulating  tools,  and  managed  its  screen  real  estate  in  a  clear  and  simple  manner. 
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II.  METHOD 


A.  MIDAS 

Mapping,  Information,  Display  and  Analysis  System  is  a  software  concept  in  re¬ 
designing  existing  LCC  user  interfaces  for  military  mission  planning  utilizing  the  new 
LCC  Software  Design  Processes  shown  in  figure  2  and  Appendix  A.  As  a  Java  based 
program,  MIDAS  takes  advantage  of  its  object-oriented  programming  by  incorporating 
existing  Java  classes  from  both  Thistle/Flora  and  SOFLCC  and  leaving  itself  open-ended 
to  the  capability  of  importing  new  methods  and  modules  via  networks  or  the  Internet. 
The  largest  contribution  MIDAS  should  make  to  the  LCC  project  is  improving  the 
usability  of  the  map-based  mission  planning.  MIDAS  specifically  addresses  the  three 
major  flaws  in  Thistle/Flora  and  SOFLCC. 

1.  Iconography 

The  icons  chosen  for  MIDAS  follow  accepted  human  factors  guidelines  for 
familiarity,  visual  and  conceptual  distinctness,  design  detail,  and  consistency  in  scheme 
(Mayhew,  1992;  DISA  1994;  DoD  1999).  The  images  were  chosen  to  help  the  user  in 
maintaining  context  and  orientation  while  reducing  the  requirements  for  memorizing 
commands  and  syntax  (Brown,  1989).  The  icons  for  each  button  are  displayed  in  table  1 
with  their  associated  functions. 
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Arrow 

Restores  mouse  to  default  features 

n 

Grab 

Enables  the  mouse  to  “grab”  the  map  and  move 
it  around  the  screen  to  re-center  or  adjust 

11 

Magnify  + 

Zooms  in  on  image  with  the  “click”  location  as 
the  new  image’s  center 

n 

Magnify  - 

Zooms  out  from  image  with  the  “click”  location 
as  the  new  image’s  center 

Text 

Brings  up  text  entry  box  to  place  text  at  the 
location  of  the  mouse  “click” 

□ 

Fade 

A  left  “click”  will  merge  the  top  image  into  the 
back  image  -  a  right  “click,”  the  opposite 

H 

Line 

Draws  a  line  from  a  “click”  and  “drag”  to  a  new 
point 

gl 

Draws  a  comer-anchored  ellipse  or  “click”- 
centered  ellipse  if  the  Shift  key  is  held 

m 

Draws  comer-anchored  rectangle  or  “click”  - 
centered  rectangle  if  the  Shift  key  is  held 

Hi 

Places  a  route  between  two  junction  images 

111 

Places  a  junction  shaped  object  on  the  image 

PI 

Enables  a  color  palette  to  choose  the  active 
color  from 

IH 

Print 

Sends  the  overlay  to  the  default  printer 

Erase 

Clears  all  overlay  objects  from  image 

SU 

Undo 

Removes  a  single  overlay  from  the  image 

a 

Restore 

Restores  a  single  overlay  to  the  image 

Table  1.  MIDAS ’s  buttons  and  their  related  functions 
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2.  Multiple  Widowing 

Unlike  Thistle/Flora  and  SOFLCC,  MIDAS  displays  one  resident  window. 
Temporary  windows  include  a  file  chooser  and  color  chooser.  The  user  may  reposition 
and  resize  the  GUI  to  fit  the  screen  as  necessary.  MIDAS  departs  from  Thistle  in  one 
other  major  feature  -  expandability.  In  Thistle,  new  OR  concepts  were  added  to  the 
program  by  adding  a  new  window  and  a  new  button  on  the  command  bar.  As  future 
features  are  added  to  MIDAS,  they  will  only  assume  “real-estate”  required  for  the 
feature’s  name  on  the  menu  bar.  The  feature’s  pull-down  menu  will  cascade  into  the 
screen  like  other  menus  and  disappear  upon  completion  of  a  specified  task.  No  additional 
resident  windows  will  be  generated  by  any  menu  item  as  in  Thistle.  Figure  7  is  a  screen 
shot  of  the  opening  view  of  MIDAS. 


Figure  7.  MIDAS  starts  with  most  map-base  tools  visible  to  the  user.  No  further  menus 

are  required  to  begin  work  on  a  map. 
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3.  Map  Manipulation 

Successful  map  manipulation  in  map-based  systems  relies  on  four  basic  points: 
image  format,  map  movement,  zoom,  and  map  annotation. 

a)  Image  Format  (.gri) 

The  LCC  concept  requires  ME) AS  to  be  as  flexible  as  possible  with  image 
formats  and  mixing  thereof.  Where  some  map-based  systems  require  consistent  use  of 
one  image  type  (i.e.  .bmp,  jpeg,  .gif,  .tif),  MIDAS’  image  handling  started  at  ground  zero. 
The  two  characteristics  of  any  map  required  for  successful  use  are  the  image  itself  and  at 
least  two  known  coordinates  on  the  image.  By  combining  the  two  into  a  geo-referenced 
image,  the  result  is  useful  for  all  map-based  planning.  MIDAS,  from  the  outset,  only 
accepts  geo-referenced  images  (.gri)  formatted  files  (Buss,  2000).  This  new  format 
solves  the  problem  of  using  any  referenced  or  non-referenced  image  type  (satellite, 
hyper-spectral,  intelligence,  topographic,  radar,  remotely  piloted  vehicle  (RPV),  hand- 
drawn,  etc.)  in  MIDAS.  Files  with  the  .gri  format  are  created  by  a  Java  class  that  imports 
an  image,  takes  latitude  and  longitude  coordinates  from  the  user,  and  serializes  them  into 
an  image  with  associated  position  tags.  Upon  importing  or  loading  the  image  into 
MIDAS,  the  position  tags  are  read  which  then  transforms  any  pixel  in  the  image  to  a 
latitude  and  longitude.  Any  annotations  then  made  to  any  image  are  ported  and  resized, 
by  latitude  and  longitude,  to  any  other  geo-referenced  image  that  may  overlap  the  same 
geographical  area.  This  image  format  also  has  exciting  possibilities  in  “rubber-sheeting” 
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non-referenced  intelligence  images  to  known  geo-referenced  images  providing  mission 
planners  another  tool  in  analyzing  images. 

b)  Map  Movement 

Map  movement  is  handled  by  two  basic  actions.  The  first  is  instantiated 
by  a  right  click  of  the  mouse  somewhere  on  the  image.  The  result  is  re-centering  the 
image  at  the  location  of  the  click. 

The  second  map  movement  is  instantiated  by  clicking  the  “Grab”  icon 
which  enables  the  mouse  controlled  drag  mode.  When  the  left  mouse  button  is  pressed  in 
this  mode,  the  image  can  be  dragged  by  movement  of  the  mouse  and  repositioned  by 
releasing  the  mouse  button. 

c)  Zoom 

Two  buttons  -  the  Zoom-In  and  Zoom-Out,  control  MIDAS’  zoom 
feature.  As  one  can  infer,  the  Zoom-In  button  replaces  the  current  image  with  one  of 
larger  scale  in  the  GUI  pane.  Zoom-Out  completes  the  opposite  function.  To  zoom  in  or 
out  on  an  image,  the  user  first  mouse-clicks  on  either  zoom-in  or  zoom-out  icon  and  then 
a  location  on  the  image.  The  place  selected  on  the  image  becomes  the  center  of  the  new 
display.  For  the  usability  study,  the  zoom  levels  were  limited  to  four  scales.  The  four 
zoom  levels  are  shown  in  figure  8.  Other  methods  for  improved  zooming  will  be  covered 
in  the  discussion  chapter. 
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Figure  8.  In  its  first  version,  MIDAS’s  zoomed  images  scale  as  determined  by  linking 
variously  scaled  maps  of  the  same  geographic  area. 

d)  Map  Annotation 

For  any  map-based  planning  software  to  be  effective,  users  must  have  the 
ability  to  annotate  it.  Thistle  gave  the  user  a  “Grease  Pen”  tool  allowing  lines,  circles, 
squares  and  symbols  to  be  overlaid  on  the  image.  The  same  capability  was  incorporated 
into  MIDAS  in  an  improved  fashion.  Accompanying  the  mentioned  functions,  are  three 
new  tools.  undo,  restore,  and  “erase.”  These  tools  allow  the  user  to  make 
corrections  to  map  annotations.  User-added  annotations  are  also  geo-referenced  to  the 
image  allowing  lines  and  marks  to  overflow  to  new  images  of  the  same  geographic  area 


and  be  resized  to  match  the  effects  of  zoomed  images. 


B.  USABILITY  EXPERIMENT 


1.  Participants 

Twenty  participants  volunteered  for  the  usability  study.  Table  2  shows  the 
breakdown  of  participants  in  the  study. 


User  Type 

USN 

USA 

USMC 

DoD 

Civilian 

— 

Totals 

Male 

7 

6 

3 

1 

1 

18 

Female 

1 

1 

2 

MAC 

1 

1 

Windows 

7 

6 

3 

1 

1 

1 

19 

TOTAL 

8 

6 

3 

1 

1 

1 

20 

Table  2.  Usability  Study  Subject  Demographics 
Sixty  percent  of  the  subjects  reported  previous  use  of  map-based  software.  All 
had  a  neutral  or  positive  attitude  toward  computer  use.  Average  computer  sessions  per 
week  were  15.2  with  an  average  session  length  of  57  minutes. 

Figures  9, 10,  and  1 1  show  computer-use  demographics  for  subject  sample. 
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2.  Apparatus 

Subjects  were  seated  in  front  of  a  standard  19”  (17.75”  viewable)  computer 
monitor  set  at  65,636  colors,  1024x768  pixels,  85Hz  refresh  rate,  and  font  size  small. 
The  processor  was  a  Pentium  IH  500  MHz  with  128  MB  RAM  and  equipped  with  a  104- 
key  standard  keyboard  and  an  Intellimouse®l.lA  PS/2. 

A  high-grade  video  recorder  with  audio  was  used  to  collect  a  record  of  each  test 
session  for  additional  information  gathering  on  each  subject.  The  video  feed  was 
broadcast  to  a  High  Definition  Television  to  enable  continuous  visual  test  monitoring. 

3.  Software 

MIDAS  was  written  in  Java  1.2.2  (Sun  Microsystems,  2000)  utilizing  Borland® 
JBuilder3®  and  various  text  editors. 

Icon  images  utilized  by  MIDAS  were  bitmaps  produced  by  the  author  using 
Microsoft’s®  Paint®  program  and  converted  to  Graphical  Interchange  File®  (.gif)  format 
using  Microsoft’s®  Image  Composer®. 

4.  Industry  HCI  Benchmarks 

Industry  benchmarks  for  usability  vary  between  companies  and  applications; 
therefore  it  is  difficult  to  find  published  success  rates  for  task  completion,  icon 
identification,  and  memorability.  In  order  to  establish  a  qualified  heuristic  benchmark, 
several  experts  in  the  HF  industry  were  consulted  for  their  opinion  as  applicable  to 
MIDAS  and  its  usability  study.  The  following  personnel  provided  a  consensus  of 
acceptable  usability  benchmarks  to  compare  MIDAS  usability  test  data.  Jose  Arcellena  is 
a  Human  Interface  Specialist  for  the  National  Broadcasting  Company  Internet  Inc.  Peter 
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Bickford  is  CEO  of  Human  Computing  Consulting  Firm  and  former  HF  specialist  at  Sun 
Microsystems.  Dr.  Mary  Cwerwinski  is  the  head  of  the  Adaptive  Systems  Interaction 
Group  at  Microsoft.  Donald  Gentner  is  Senior  Staff  Engineer  and  Human  Interface 
Designer  at  JavaSoft,  Sun  Microsystems.  John  Pane  is  a  Graduate  Assistant  to  the 
Computer  Science  Department,  Carnegie  Mellon  University. 

5.  Procedure 

Subjects  were  asked  to  read  and  sign  various  consent  and  experiment  information 
forms  as  well  as  a  background  questionnaire  (Appendix  B)  based  on  Rubin’s 
questionnaire  for  computer  usability  studies  (1994).  Data  gathered  during  the  experiment 
was  annotated  on  the  Data  Collection  Sheet  found  in  Appendix  C. 

After  all  forms  were  completed,  subjects  were  moved  to  a  half-walled  cubicle 
where  the  software  was  already  running  and  seated  at  the  computer  station.  They  were 
then  asked  to  describe  the  function,  by  iconography  only,  of  selected  buttons  annotated  in 
Appendix  C.  Subjects  were  then  asked  to  enable  the  tool  tips  option  and  review  the 
function  of  any  button  they  were  unsure  of.  Subjects  were  then  asked  to  complete  nine 
more  pre-determined  tasks  also  shown  in  Appendix  C.  Each  of  the  tasks  addressed  the 
user’s  ability  to  complete  map-manipulating  functions  with  no  prior  training  or  exposure 
to  the  interface. 

Each  experiment  session  lasted  approximately  15  minutes.  Upon  completion  of 
the  tasks,  subjects  were  allowed  to  ask  questions  to  clarify  any  difficulties  and  make 
comments  regarding  the  program’s  interface  or  future  use. 
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To  test  for  Memorability  of  MIDAS’  GUI,  ten  subjects  received  an  icon 
recognition  test  (Appendix  C)  approximately  one  week  after  their  initial  testing.  Subjects 
were  asked  to  provide  the  function  of  the  same  buttons  that  were  tested  in  the  initial  test. 

User  comments  and  recommendations  for  improved  icons  were  annotated  and  are 
discussed  in  the  Results. 
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III.  RESULTS 


A.  LEARNABILITY 

Leamability  was  tested  using  two  methods.  The  first  was  straight  identification 
rates  compared  to  industry  standards.  The  second  method  determined  that  MIDAS’s  icon 
identification  rates  are  predictable. 

1.  Identification  Rates 

The  industry  standard  for  icon  identification  rates  is  70%  for  initial  contact 
(Czerwinski,  personal  communication,  1 1  May  2000,  Bickford,  personal  communication, 
11  May  2000).  This  standard  was  used  to  establish  which  icons  in  ME) AS’  GUI  required 
redesign.  Figure  12  shows  the  icon  identification  rates  for  all  twenty  subjects  across  all 
thirteen  icons. 


Figure  12.  Icon  Identification  Rates  (n=20) 
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The  four  icons  identified  as  having  poor  leamability  are  Text,  Erase,  Undo  and 
Restore,  each  of  which  scored  below  the  70%  identification  rate.  The  most  common  icon 
misidentifications  or  responses  for  the  four  icons  are  summarized  in  Table  3. 


Button 

Response 

Rate 

Text 

“Don’t  know” 

30% 

Erase 

“3-D  drawing” 

50% 

Undo 

“Rotate  Left  90°” 

70% 

Restore 

“Rotate  Right  90°” 

70% 

Table  3.  Misidentification  Responses  and  Associated  Rates 


The  icons’  poor  leamability  may  be  attributed  to  two  reasons.  First,  they  did  not 
provide  enough  visual  cues  to  the  user  to  establish  a  metaphor  for  their  use.  Second,  they 
matched  too  closely  to  icons  in  other  software  that  perform  very  different  functions. 
Third,  they  did  not  match  icons  from  other  software  that  perform  the  intended  functions. 
The  four  poor  icons  should  be  redesigned  and  re-tested  utilizing  the  study’s  feedback  and 
compared  to  benchmarks  established. 

2.  Icon  Identification  Predictability 

Leamability  was  also  examined  to  determine  if  subjects’  scores  were  dependent 
upon  his/her  demographics.  A  least  squares  regression  model  was  calculated  to 
determine  if  subjects’  scores  for  icon  identification  were  influenced  by  demographics 
(Agresti,  1990,  Cook  &  Weisberg,  1999).  Subject’s  scores  were  determined  to  be 
dependent  upon  demographic  data  via  the  following  statistically  significant  model  with 
an  alpha  level  of  .05  (F  (3,15)  =  4.03,  p  =  .0275).  These  results  show  that  icon 
identification  for  a  similarly  demographic  sample  can  be  predicted  by  knowing  the 
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average  length  of  their  computer  session,  how  many  sessions  a  week,  and  number  of 
years  computer  experience. 

Score  =  4.6410  +  (.0173)x  [Length)- (.6045)x  [Session)+  (.5930)(Yetf  rs) 

The  regression  model  was  validated  using  the  metrics  and  test  statistics  shown  in 
Table  4.  Supporting  graphs  are  in  Appendix  E. 


Regression 

p  =  .0275 

R2" 

Non-Constant  Variance 

p  =  .303 

Curvature  [fitted  values] 

p  =  .243 

Curvature  [Length] 

p  =  .957 

Curvature  [Session] 

p  =  .772 

Curvature  [Years] 

p  =  .961 

Table  4.  Regression  Statistics 

These  results  are  important  for  two  reasons.  First,  they  show  future  interface 
designers  the  demographics  that  determined  successful  identification  in  this  sample 
population.  Second,  they  provide  the  designer  guidance  to  tailor  the  interface  for  a  target 
population  or  take  steps  to  broaden  the  identification  rate  across  a  larger  demographically 
diverse  sample. 
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B.  MEMORABILITY 


The  metric  for  determining  Memorability  was  a  second  icon  identification  test 
one  week  after  the  initial  exposure  to  the  icons.  The  acceptable  industry  success  rate  for 
Memorability  is  100%  (Gentner,  personal  communication,  12  May  2000,  Arcellena, 
personal  communication,  12  May  2000,  Pane,  personal  communication  12  May  2000). 


CO 

o 


c 

0) 

2 

*-• 

o 

o 


o 

o 

c 

© 

2 

a> 

Q. 


Figure  13.  Icon  identification  success  rate  for  Memorability 

The  majority  of  MIDAS’  icons  scored  well  in  Memorability.  After  only  one 
exposure  to  the  icons,  10  of  13  icon  recognition  rates  were  100%  when  tested  one  week 
later.  Icons  with  problems,  Erase,  Undo,  and  Restore,  were  well  below  the  standard  of 
100%.  Their  lower  rates  are  due  to  the  same  reasons  they  had  poor  Leamability.  The 
icons  were  confused  with  icons  already  learned  from  other  software  that  complete  very 
different  functions  than  those  in  MIDAS.  After  three  new  icons  are  shown  with 
acceptable  Leamability,  future  studies  should  test  shorter  and  longer  in-between-use 
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times.  These  results  show  the  majority  of  images  selected  for  the  button  bar  are  easy  to 
remember  even  without  consistent  use. 

C.  EFFICIENCY 

Overall,  MIDAS  has  an  efficient  interface.  With  no  prior  training,  subjects  are 
able  to  quickly  become  productive  scoring  above  90%  for  all  map  manipulation  tasks. 
Figure  14  shows  success  rates  for  the  ten  map  manipulation  tasks  described  in  Appendix 
C. 
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Figure  14.  Map  Manipulation  Task  Success  Rates  (n=20) 


Though  users  could  not  identify  the  function  of  all  of  the  icons  with  the  tool  tips 
feature  disabled,  they  were  able  to  successfully  complete  the  tasks  with  tool  tips  enabled. 
This  result  shows  the  success  of  the  interface  design  concept  that  gives  the  user  the 
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ability  to  add  helpful  information  to  the  interface  to  improve  efficiency.  The  ability  to 
turn  the  tool  tips  off  also  allows  experienced  users  eliminate  possibly  annoying  clutter. 

Future  work  in  Efficiency  should  focus  on  an  in-depth  task  analysis  or  scenario 
driven  task  list.  Many  subjects  did  request  to  use  the  software  for  personal  or  school- 
related  work  -  a  subjective  sign  that  the  interface  was  easy  to  learn  and  use. 
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IV.  DISCUSSION 


The  goals  defined  at  the  onset  of  this  thesis  were  to  produce  a  quality  easy-to-use 
graphic  user  interface  (GUI)  for  map-based  mission  planning  and  conduct  a  usability  test 
to  determine  its  design  success.  The  usability  study  identified  some  weaknesses  in  icon 
design  that  in  the  next  version  of  MIDAS  will  be  improved  and  re-tested.  The  weakness 
were  not  strong  enough  to  adversely  affect  the  overall  success  of  MIDAS’s  interface. 

MIDAS  successfully  combines  many  of  the  proven  tools  from  existing  Loosely 
Coupled  Components  software  into  one  streamlined  design  while  incorporating  the  strong 
design  points  of  established  human  factors  guidelines.  With  continued  GUI  improvement 
and  testing,  MIDAS  will  grow  to  become  a  powerful  and  portable  map-based  mission¬ 
planning  tool.  Recommended  improvements  in  MIDAS’  interface  can  now  be  tested 
against  the  benchmark  established  in  this  thesis. 

A.  RECOMMENDATIONS 

MIDAS  will  continue  to  evolve  under  the  guidance  of  the  LCC  research  group. 
As  new  map-based  Operations  Research  tools  are  developed,  they  must  be  subjected  to 
the  new  Software  Design  Process  and  must  comply  with  established  DoD  (1999),  DISA 
(1994)  and  Industry  HCI  standards.  Technology  will  continue  to  evolve  bringing  smaller 
displays  and  unique  pointing  devices  to  the  doorstep  of  LCC.  The  research  group  must 
maintain  an  understanding  of  the  human  factors  involved  to  successfully  exploit  the 
capabilities  of  these  emerging  technologies. 
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APPENDIX  A.  LCC  SOFTWARE  DESIGN  PROCESS 
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APPENDIX  B.  MIDAS  SUBJECT  QUESTIONNAIRE 


1.  What  is  your  age?  (20-25)  (25-30)  (30-35)  (>35) 

2.  Male  or  Female?  M  F 

3.  Occupation?  _ 

4.  If  military,  what  rank  and  branch?  _ 

5.  Highest  Grade  Completed? 

12  Assoc.  BA/BS  MA/MS  MD/Ph.D. 

6.  Which  is  your  dominate  hand?  Left  Right 

7.  Are  you  currently  experiencing  any  problems  that  impair  your  ability  to  use  a 
computer? 

a)  Yes  b)  No 

If  yes,  what  are  they?  _ 

8.  How  many  times  do  you  use  a  computer  a  week?  1-5  5-10  10-15 

9.  What  is  your  most  common  computing  session  length? 

<10min  10-30min  30-60  min  60-90min  >90min 

10.  How  many  sessions  of  this  type  do  you  have  a  day? 

a)  1 

b)  2 

c)  3 

d)  4 

e)  >4 


>15 
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1 1.  Which  of  the  following  applications  do  you  most  often  use  on  a  daily  basis?  (circle  as 
many  as  necessary) 


a)  Send  /  Receive  e-mail 

b)  Surf  the  Internet 

c)  Word  Processing 

d)  Finances 

e)  Spreadsheets 

f)  Games 

g)  Presentations 

h)  Programming 

i)  Other _ _ _ 

12.  What  operating  system  do  you  primarily  use?  (circle  more  than  one  if  needed) 

a)  Windows  9X,  NT,  2K 

b)  Mac 

c)  Linux 

d)  Unix 

13.  How  many  years  have  you  been  actively  using  a  computer? 

a)  <  1 

b)  1-3 

c)  3-5 

d)  5-9 

e)  >  10  yrs. 

14.  Have  you  used  map-based  software?  (commercial,  military,  Internet,  etc.) 

a)  Yes 

b)  No 

15.  Are  you  geographically  familiar  with  the  Monterey  Peninsula? 

a)  Yes 

b)  No 

16.  What  is  your  attitude  toward  computer  use? 

a)  Positive 

b)  Indifferent 

c)  Negative 
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APPENDIX  C.  MIDAS  DATA  COLLECTION  SHEET 


PART  1:  ICONOGRAPHY 


“What  do  you  think  the  functions  of  the  following  icons  are? 


fiRS 

il|| 

iim 

~~  ~  -  -v- ^  ;  •  •••  : 

Bis 

pointer,  arrow,  mouse  control 

H9 

grab,  drag,  move 

ft. 

zoom  in,  make  larger 

E9 

zoom  out,  make  smaller 

El 

add  text,  annotate,  add  words,  label 

iHt 

jj  M  Wm 

draw  line 

o 

draw  circle,  draw  ellipse 

EO 

draw  box,  draw  rectangle 

mm 

sgiSHS 

||J|lggg 

IfllPl 

H^Htg 

change  color 

m 

print,  print  image,  print  map 

erase,  remove 

n 

undo,  undo  last 

n 

restore,  restore  last,  return,  undo  an  undo 
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PART  2:  MAP  MANIPULATION 


Pi 

■1 

jjjgjj 

illlijj 

Options  Menu 

■ 

Zoom  In 

“Zoom  in  until  you  find  Naval 
Postgraduate  School.” 

Draw  Box 

“Draw  a  box  around  the  entire 
School.” 

Grab 

“Grab  the  map  and  re-center  to 
place  wharf  area  in  the  center  of 
the  screen.” 

Zoom 

“Zoom  in  on  the  Northern 
Wharf.” 

“Change  active  color  to  red.” 

Draw  Circle 

“Draw  a  circle  around  the  end 
of  the  pier.” 

Zoom  Out 

“Zoom  out  until  you  see  both 
Lover’s  Point  and  the 

Municipal  Airport.” 

“Draw  a  line  between  the 
airport  and  Lover’s  Point.” 

Insert  Text 

“Label  the  Monterey  Bay.” 

Lat  /  Long 

Readout 

“What  is  the  Lat  /  Long  of 
Herman  Hall?” 
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APPENDIX  D.  FOLLOW-UP  ICON  RECOGNITION  TEST 


MIDAS  v.  1  Usability  Test  Data  Collection  Sheet 

Part  Two 


In  two  or  three  words,  what  do  you  think  the  functions  of  the  following  icons  are? 


Icon 

Answer 
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APPENDIX  E.  REGRESSION  SUPPORT 
Score  =  4.6410  +  (.0173)x  (Length)- (.6 045)x  (Session)  +  (,5930)(  Years) 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


54 


LIST  OF  REFERENCES 


Agresti,  A,  (1990).  Categorical  Data  Analysis.  John  Wiley  &  Sons,  Inc.,  New  York,  New 
York. 

Bilyeu,  A.  L.  (1998).  Concept  for  a  Special  Operations  Planning  and  Analysis  System. 
Master’s  Thesis,  Naval  Postgraduate  School,  Monterey,  California. 

Bradley,  G.,  Buss,  A.  &  Shaw  (1998).  An  Architecture  for  Dynamic  Planning  and 
Execution  Using  Loosely  Coupled  Components.  Naval  Postgraduate  School  Research 
Letter,  8, 1-7. 


Brown,  C.M.  (1989).  Human-Computer  Interface  Design  Guidelines.  Ablex  Publishing 
Corporation,  Norwood,  New  Jersey. 

Buss,  A.  H.  (2000).  A  Note  on  Geo-Referenced  Images,  Unpublished  manuscript. 
Department  of  Operations  Research,  Naval  Postgraduate  School,  Monterey,  CA. 

Card,  S.,  Moran,  T.  P.  &  Newell,  A.  (1983)  The  psychology  of  human-computer 
interaction.  Lawrence  Erlbaum  Associates,  Hillsdale,  New  Jersey. 

Chairman  of  the  Joint  Chiefs  of  Staff  (1996).  Joint  Vision  2010. ,  Pentagon,  Washington, 
D.C.:  Author. 

Defense  Information  Systems  Agency  (1994).  Department  of  Defense  Technical 
Architecture  Framework  for  Information  Management.  Volume  8:  Department  of 
Defense  HCI  Style  Guide.  (v2.0,  30  June  1994). 

Department  of  Defense  (1999).  Design  Criteria  Standard:  Human  Engineering.  (DoD 
Publication  No.  MIL-STD-1472F),  Washington,  D.C.:  Author. 

Despain,  A.,  &  Westervelt,  R.  (1997).  High  Performance  Human-Computer  Interfaces. 
Project  Summary  Booklet,  JASON  Program  Office,  The  MITRE  Corporation,  sponsored 
by  DARPA. 

Devore,  J.  L.  (1995).  Probability  and  Statistics  for  Engineering  and  the  Sciences.  4th  ed. 
Brooks/Cole  Publishing  Company,  Pacific  Grove,  California. 

Dix,  A.  J.,  Finlay,  J.  E.,  Abowd,  G.  D.  &  Beale,  R.  (1998).  Human-Computer 
Interaction,  Prentice  Hall  Europe. 


55 


Halasz,  F.,  &  Moran,  T.  P.  (1982).  Analogy  Considered  Harmful,  Proceedings  of 
Human  Factors  in  Computer  Systems,  pp.  383-386,  Washington,  D.C.:  National  Bureau 
of  Standards. 

Hattes,  K.  A.  (1999).  Special  Operations  Mission  Planning  and  Analysis  Support 
System,  Master’s  Thesis,  Naval  Postgraduate  School,  Monterey,  California. 

Hefley,  W.  E.,  Buie,  E.  A.,  Lynch,  G.  F.,  Muller,  M.  J.,  Hoecker,  D.  G.,  Carter,  J.,  & 
Roth,  J.  T.  (1994).  Integrating  Human  Factors  With  Software  Engineering  Practices, 
Proceedings  of  the  Human  Factors  and  Ergonomic  Society.  38.  315-323. 

Hix,  D.  &  Hartson,  R.  H.  (1993).  Developing  User  Interfaces  Ensuring  Usability 
Through  Product  &  Process.  John  Wiley  &  Sons,  Inc.,  New  York,  New  York. 

Jette.  (1999).  Land  Warrior  Technology  Interests.  MS  Power  Point  Presentation  SY 
Technology  Inc.,  U.S.  Army  Soldier  Systems  Center,  Natick,  MA. 

Lim,  K.  &  Long,  J.  (1992).  A  Method  for  (Recruiting)  Methods:  Facilitating  Human 
Factors  Input  to  System  Design,  Proceedings  from  Computer-Human  Interaction  CHI  92. 
5,  549-556. 

Linden,  P.  van  der.  (1997).  NotJustJava,  Sun  Microsystems  Inc.,  Mountain  View, 
California. 

Lund,  A.,  &  Tschirgi,  J.  (1991).  Designing  for  People:  Integrating  Human  Factors  into 
the  Product  Realization  Process,  IEEE  Journal  on  Selected  Areas  in  Communications  9 
496-500. 

Mayhew,  D.  J.  (1992).  Principles  and  Guidelines  in  Software  User  Interface  Design. 
Prentice  Hall,  Englewood  Cliffs,  New  Jersey. 

Microsoft  (1998).  Microsoft  Usability:  Making  It  Easier  for  Everyone.  White  Paper, 
[http://www.microsoft.eom/PressPass/features/l  998/5-  19usewp.asp] . 

Minasi,  M.  (1994).  Secrets  of  Effective  GUI  Design.  SYBEX,  Alameda,  California. 

Nielsen,  J.  (1999).  Heuristic  Evaluation.  Usability  Inspection  Methods,  John  Wiley  & 
Sons,  New  York,  New  York. 

Nielsen,  J.  (1993).  Usability  Engineering.  Academic  Press.  Cambridge  Masgarhngpttg 

Nielsen,  J.  &  Molich,  R.  (1990).  Improving  a  Human-Computer  Dialog, 

Communications  of  the  ACM.  33,  338-348. 


56 


Nolan,  P.  R.  (1989).  Designing  Screen  Icons:  Ranking  and  Matching  Studies, 
Proceedings  of  the  Human  Factors  Society  33rd  Annual  Meeting.  33, 380-389. 

Page,  W.  (2000).  Command  Post  of  the  Future  (CPOF)  Program  Description,  (DARPA 
Publication  No.  BAA-98-27),  Washington,  D.C.:  U.S.  Government  Printing  Office. 

Raskin,  J.  (1997).  Looking  for  a  humane  interface:  Will  computers  ever  become  easy  to 
use?,  Association  for  Computing  Machinery,  Communications  of  the  ACM.  40, 98-101. 

Reiterer,  Dr.  H.  (1993).  The  Development  of  Design  Aid  Tools  for  a  Human  Factor 
Based  User  Interface  Design,  Proceedings  of  the  International  Conference  on  System 
Engineering  in  the  Service  of  Humans.  4,  361-366. 

Rubin,  J.  (1994).  Handbook  of  Usability  Testing.  How  to  Plan.  Design,  and  Conduct 
Effective  Tests.  John  Wiley  &  Sons,  Inc.  New  York,  New  York. 

Schrepf,  N.  J.  (1999).  Visual  Planning  Aid  for  Movement  of  Ground  Forces  in 
Operations  Other  Than  War,  Master’s  Thesis,  Naval  Postgraduate  School,  Monterey, 
California. 

Shneiderman,  B.  (1998).  Designing  the  User  Interface  -  Strategies  for  Effective  Human- 
Computer  Interaction,  Third  Edition,  Addison- Wesley  Longman  Inc.,  Menlo  Park, 
California. 

Sun  Microsystems  (2000).  Java  Development  Kit  v  1.2.2, 
[http://java.sun.eom/products/jdkl.2/]. 

Wickens,  C.  D.,  Gordon,  S.  E.,  &  Liu,  Y.  (1998).  An  Introduction  to  Human  Factors 
Engineering.  Addison-Wesley  Education  Publishers  Inc.,  Menlo  Park,  California. 

Wozny,  L.A.  (1989).  The  application  of  metaphor,  analogy,  and  conceptual  models  in 
computer  systems.  Interacting  with  Computers,  1,  273-283. 


57 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


58 


INITIAL  DISTRIBUTION  LIST 


1.  Defense  Technical  Information  Center . 2 

8725  John  J.  Kingman  Rd.,  STE  0944 

Ft.  Belvoir,  Virginia  22060-6218 

2.  Dudley  Knox  Library . 2 

Naval  Postgraduate  School 

411  Dyer  Rd. 

Monterey,  CA  93943-5101 

3.  Professor  Gordon  H.  Bradley . 2 

Department  of  Operations  Research 

Naval  Post  Graduate  School 
Monterey,  CA  93943-5000 

4.  Professor  William  K.  Krebs . 2 

Department  of  Operations  Research 

Naval  Post  Graduate  School 
Monterey,  CA  93943-5000 

5.  Professor  Arnold  H.  Buss . 1 

Department  of  Operations  Research 

Naval  Post  Graduate  School 
Monterey,  CA  93943-5000 

6.  Professor  Rudolph  P.  Darken . 1 

Department  of  Computer  Science 

Naval  Post  Graduate  School 
Monterey,  CA  93943-5000 

7.  Major  Leroy  A.  Jackson . 1 

TRADOC  Analysis  Center  -  Monterey 

Naval  Postgraduate  School,  P.O.  Box  8692 
Monterey,  CA  93940-5000 

8.  LT  James  P.  Lowell . 4 

c/o  HMCM  James  M.  Lowell  (ret.) 

1459  Red  Oak  Lane 
Port  Charlotte,  FL  33948 


59 


